Aiming at the poor effect of traditional machine learning algorithms on large data volume shipping monitoring video recognition classification and the low recognition accuracy of previous three-Dimensional (3D) convolution, based on 3D convolutional neural network model, combined with the popular Visual Geometry Group (VGG) network structure and GoogleNet's Inception network structure, a new VGG-Inception 3D Convolutional neural network (VIC3D) model based on VGG-16 3D convolutional network and introduced Inception module was proposed to realize the intelligent recognition of the real-time monitoring video of shipping goods. Firstly, the video data acquired from the camera were processed into images. Then, the video frame sequences by equal interval frame fetching were classified according to the categories, and the training set and the testing set were constructed. Under the premise of the same operating environment and the same training mode, the VIC3D model after combination and the original model were trained separately. Finally, the various models were compared based on the test results of the testing set. The experimental results show that, compared with the original model, the recognition accuracy of VIC3D model is improved, which is increased by 11.1 percentage points compared to the Group-constrained Convolutional Recurrent Neural Network (GCRNN) model, and the time required for every recognition is reduced by 1.349 s; the recognition accuracy of VIC3D model is increased by 14.6 percentage points and 4.2 percentage points respectively compared to the two models of C3D. The VIC3D model can be effectively applied to the shipping video surveillance projects.